Paper-based versus computer-based assessment: key factors associated with the test mode effect

This investigation seeks to confirm several key factors in computer-based versus paper-based assessment. Based on earlier research, the factors considered here include content familiarity, computer familiarity, competitiveness, and gender. Following classroom instruction, freshman business undergraduates (N = 105) were randomly assigned to either a computer-based or identical paper-based test. ANOVA of test data showed that the computer-based test group outperformed the paper-based test group. Gender, competitiveness, and computer familiarity were NOT related to this performance difference, though content familiarity was. Higher-attaining students benefited most from computer-based assessment relative to higher-attaining students under paper-based testing. With the current increase in computer-based assessment, instructors and institutions must be aware of and plan for possible test mode effects. As pointed out by other authors in this special issue, for many reasons, the use of computer-based assessment is increasing. Examples include state drivers’ license exams, military training exams, job application exams in the private sector, entrance exams in postsecondary education, and certification exams by professional groups (Russo, 2002; Trotter, 2001). However, in the literature, there is mounting empirical evidence that identical paper-based and computer-based tests will not obtain the same results. Such findings are referred to as the “test mode effect.” For example, paper-based test scores were greater than computer-based test scores for both mathematics and English CLEP tests (Mazzeo, Druesne, Raffeld, Checketts and Muhlstein, 1991) and for recognizing fighter plane silhouettes (Federico, 1989); while computer-based test scores were greater than paper-based test scores for a dental hygiene course unit midterm examination (DeAngelis, 2000); though other studies 08_Clar 15/10/02 12:41 pm Page 593